<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta name="generator" content="HTML Tidy, see www.w3.org" />

    <title>Apache 1.3 Dynamic Shared Object (DSO) support</title>
  </head>
  <!-- Background white, links blue (unvisited), navy (visited), red (active) -->

  <body bgcolor="#FFFFFF" text="#000000" link="#0000FF"
  vlink="#000080" alink="#FF0000">
    <blockquote>
      <!--#include virtual="header.html" -->

      <div align="CENTER">
        <h1>Apache 1.3<br />
         Dynamic Shared Object (DSO)<br />
         Support</h1>

        <address>
          Originally written by<br />
           Ralf S. Engelschall &lt;rse@apache.org&gt;, April 1998
        </address>
      </div>

      <h3>Background</h3>

      <p>On modern Unix derivatives there exists a nifty mechanism
      usually called dynamic linking/loading of <em>Dynamic Shared
      Objects</em> (DSO) which provides a way to build a piece of
      program code in a special format for loading it at run-time
      into the address space of an executable program.</p>

      <p>This loading can usually be done in two ways:
      Automatically by a system program called <code>ld.so</code>
      when an executable program is started or manually from within
      the executing program via a programmatic system interface to
      the Unix loader through the system calls
      <code>dlopen()/dlsym()</code>.</p>

      <p>In the first way the DSO's are usually called <em>shared
      libraries</em> or <em>DSO libraries</em> and named
      <code>libfoo.so</code> or <code>libfoo.so.1.2</code>. They
      reside in a system directory (usually <code>/usr/lib</code>)
      and the link to the executable program is established at
      build-time by specifying <code>-lfoo</code> to the linker
      command. This hard-codes library references into the
      executable program file so that at start-time the Unix loader
      is able to locate <code>libfoo.so</code> in
      <code>/usr/lib</code>, in paths hard-coded via linker-options
      like <code>-R</code> or in paths configured via the
      environment variable <code>LD_LIBRARY_PATH</code>. It then
      resolves any (yet unresolved) symbols in the executable
      program which are available in the DSO.</p>

      <p>Symbols in the executable program are usually not
      referenced by the DSO (because it's a reusable library of
      general code) and hence no further resolving has to be done.
      The executable program has no need to do anything on its own
      to use the symbols from the DSO because the complete
      resolving is done by the Unix loader. (In fact, the code to
      invoke <code>ld.so</code> is part of the run-time startup
      code which is linked into every executable program which has
      been bound non-static). The advantage of dynamic loading of
      common library code is obvious: the library code needs to be
      stored only once, in a system library like
      <code>libc.so</code>, saving disk space for every
      program.</p>

      <p>In the second way the DSO's are usually called <em>shared
      objects</em> or <em>DSO files</em> and can be named with an
      arbitrary extension (although the canonical name is
      <code>foo.so</code>). These files usually stay inside a
      program-specific directory and there is no automatically
      established link to the executable program where they are
      used. Instead the executable program manually loads the DSO
      at run-time into its address space via <code>dlopen()</code>.
      At this time no resolving of symbols from the DSO for the
      executable program is done. But instead the Unix loader
      automatically resolves any (yet unresolved) symbols in the
      DSO from the set of symbols exported by the executable
      program and its already loaded DSO libraries (especially all
      symbols from the ubiquitous <code>libc.so</code>). This way
      the DSO gets knowledge of the executable program's symbol set
      as if it had been statically linked with it in the first
      place.</p>

      <p>Finally, to take advantage of the DSO's API the executable
      program has to resolve particular symbols from the DSO via
      <code>dlsym()</code> for later use inside dispatch tables
      <em>etc.</em> In other words: The executable program has to
      manually resolve every symbol it needs to be able to use it.
      The advantage of such a mechanism is that optional program
      parts need not be loaded (and thus do not spend memory) until
      they are needed by the program in question. When required,
      these program parts can be loaded dynamically to extend the
      base program's functionality.</p>

      <p>Although this DSO mechanism sounds straightforward there
      is at least one difficult step here: The resolving of symbols
      from the executable program for the DSO when using a DSO to
      extend a program (the second way). Why? Because "reverse
      resolving" DSO symbols from the executable program's symbol
      set is against the library design (where the library has no
      knowledge about the programs it is used by) and is neither
      available under all platforms nor standardized. In practice
      the executable program's global symbols are often not
      re-exported and thus not available for use in a DSO. Finding
      a way to force the linker to export all global symbols is the
      main problem one has to solve when using DSO for extending a
      program at run-time.</p>

      <p>Windows and NetWare provide similar facilities, although
      they are implemented somewhat differently than the
      description of Unix DSO throughout this document. In
      particular, DSO modules (DLL's and NLM's, respectively) are
      built quite differently than their Unix cousins. This
      document does not attempt to explore the topic of building
      DSO modules on these platforms. The description of mod_so and
      its configuration, however, are similar.</p>

      <h3>Practical Usage</h3>

      <p>The shared library approach is the typical one, because it
      is what the DSO mechanism was designed for, hence it is used
      for nearly all types of libraries the operating system
      provides. On the other hand using shared objects for
      extending a program is not used by a lot of programs.</p>

      <p>As of 1998 there are only a few software packages
      available which use the DSO mechanism to actually extend
      their functionality at run-time: Perl 5 (via its XS mechanism
      and the DynaLoader module), Netscape Server, <em>etc.</em>
      Starting with version 1.3, Apache joined the crew, because
      Apache already uses a module concept to extend its
      functionality and internally uses a dispatch-list-based
      approach to link external modules into the Apache core
      functionality. So, Apache is really predestined for using DSO
      to load its modules at run-time.</p>

      <p>As of Apache 1.3, the configuration system supports two
      optional features for taking advantage of the modular DSO
      approach: compilation of the Apache core program into a DSO
      library for shared usage and compilation of the Apache
      modules into DSO files for explicit loading at run-time.</p>

      <h3>Implementation</h3>

      <p>The DSO support for loading individual Apache modules is
      based on a module named <a
      href="mod/mod_so.html"><code>mod_so.c</code></a> which has to
      be statically compiled into the Apache core. It is the only
      module besides <code>http_core.c</code> which cannot be put
      into a DSO itself (bootstrapping!). Practically all other
      distributed Apache modules can then be placed into a DSO
      by individually enabling the DSO build for them via
      <code>configure</code>'s <code>--enable-shared</code> option
      (see top-level <code>INSTALL</code> file) or by changing the
      <code>AddModule</code> command in your
      <code>src/Configuration</code> into a
      <code>SharedModule</code> command (see
      <code>src/INSTALL</code> file). After a module is compiled
      into a DSO named <code>mod_foo.so</code> you can use <a
      href="mod/mod_so.html"><code>mod_so</code></a>'s <a
      href="mod/mod_so.html#loadmodule"><code>LoadModule</code></a>
      command in your <code>httpd.conf</code> file to load this
      module at server startup or restart.</p>

      <p>To simplify this creation of DSO files for Apache modules
      (especially for third-party modules) a new support program
      named <a href="programs/apxs.html">apxs</a> (<em>APache
      eXtenSion</em>) is available. It can be used to build DSO
      based modules <em>outside of</em> the Apache source tree. The
      idea is simple: When installing Apache the
      <code>configure</code>'s <code>make install</code> procedure
      installs the Apache C header files and puts the
      platform-dependent compiler and linker flags for building DSO
      files into the <code>apxs</code> program. This way the user
      can use <code>apxs</code> to compile his Apache module
      sources without the Apache distribution source tree and
      without having to fiddle with the platform-dependent compiler
      and linker flags for DSO support.</p>

      <p>To place the complete Apache core program into a DSO
      library (only required on some of the supported platforms to
      force the linker to export the apache core symbols -- a
      prerequisite for the DSO modularization) the rule
      <code>SHARED_CORE</code> has to be enabled via
      <code>configure</code>'s
      <code>--enable-rule=SHARED_CORE</code> option (see top-level
      <code>INSTALL</code> file) or by changing the
      <code>Rule</code> command in your <code>Configuration</code>
      file to <code>Rule SHARED_CORE=yes</code> (see
      <code>src/INSTALL</code> file). The Apache core code is then
      placed into a DSO library named <code>libhttpd.so</code>.
      Because one cannot link a DSO against static libraries on all
      platforms, an additional executable program named
      <code>libhttpd.ep</code> is created which both binds this
      static code and provides a stub for the <code>main()</code>
      function. Finally the <code>httpd</code> executable program
      itself is replaced by a bootstrapping code which
      automatically makes sure the Unix loader is able to load and
      start <code>libhttpd.ep</code> by providing the
      <code>LD_LIBRARY_PATH</code> to <code>libhttpd.so</code>.</p>

      <h3>Supported Platforms</h3>

      <p>Apache's <code>src/Configure</code> script currently has
      only limited but adequate built-in knowledge on how to
      compile DSO files, because as already mentioned this is
      heavily platform-dependent. Nevertheless all major Unix
      platforms are supported. The definitive current state (May
      1999) is this:</p>

      <ul>
        <li>
          Out-of-the-box supported platforms:<br />
           (actually tested versions in parenthesis) 
<pre>
o  FreeBSD            (2.1.5, 2.2.x, 3.x, 4.x)
o  OpenBSD            (2.x)
o  NetBSD             (1.3.1)
o  BSDI               (3.x, 4.x)
o  Linux              (Debian/1.3.1, RedHat/4.2)
o  Solaris            (2.4, 2.5, 2.6, 2.7)
o  SunOS              (4.1.3)
o  Digital UNIX       (4.0)
o  IRIX               (5.3, 6.2)
o  HP/UX              (10.20)
o  UnixWare           (2.01, 2.1.2)
o  SCO                (5.0.4)
o  AIX                (3.2, 4.1.5, 4.2, 4.3)
o  ReliantUNIX/SINIX  (5.43)
o  SVR4               (-)
o  Mac OS X Server    (1.0)
o  Mac OS             (10.0 preview 1)
o  OpenStep/Mach      (4.2)
o  DGUX               (??)
o  NetWare            (5.1)
o  Windows            (95, 98, NT 4.0, 2000)
</pre>
        </li>

        <li>
          Explicitly unsupported platforms: 
<pre>
o  Ultrix             (no dlopen-style interface under this platform)
</pre>
        </li>
      </ul>

      <h3>Usage Summary</h3>

      <p>To give you an overview of the DSO features of Apache 1.3,
      here is a short and concise summary:</p>

      <ol>
        <li>
          Placing the Apache core code (all the stuff which usually
          forms the <code>httpd</code> binary) into a DSO
          <code>libhttpd.so</code>, an executable program
          <code>libhttpd.ep</code> and a bootstrapping executable
          program <code>httpd</code> (Notice: this is only required
          on some of the supported platforms to force the linker to
          export the Apache core symbols, which in turn is a
          prerequisite for the DSO modularization): 

          <ul>
            <li>
              Build and install via <code>configure</code>
              (preferred): 

              <table bgcolor="#f0f0f0" cellpadding="10">
                <tr>
                  <td>
<pre>
$ ./configure --prefix=/path/to/install
              --enable-rule=SHARED_CORE ...
$ make install
</pre>
                  </td>
                </tr>
              </table>
            </li>

            <li>
              Build and install manually: 

              <table bgcolor="#f0f0f0" cellpadding="10">
                <tr>
                  <td>
<pre>
- Edit src/Configuration:
  &lt;&lt; Rule SHARED_CORE=default
  &gt;&gt; Rule SHARED_CORE=yes
  &lt;&lt; EXTRA_CFLAGS= 
  &gt;&gt; EXTRA_CFLAGS= -DSHARED_CORE_DIR=\"/path/to/install/libexec\"
$ make 
$ cp src/libhttpd.so* /path/to/install/libexec/
$ cp src/libhttpd.ep  /path/to/install/libexec/
$ cp src/httpd        /path/to/install/bin/
</pre>
                  </td>
                </tr>
              </table>
            </li>
          </ul>
        </li>

        <li>
          Build and install a <em>distributed</em> Apache module,
          say <code>mod_foo.c</code>, into its own DSO
          <code>mod_foo.so</code>: 

          <ul>
            <li>
              Build and install via <code>configure</code>
              (preferred): 

              <table bgcolor="#f0f0f0" cellpadding="10">
                <tr>
                  <td>
<pre>
$ ./configure --prefix=/path/to/install
        --enable-shared=foo
$ make install
</pre>
                  </td>
                </tr>
              </table>
            </li>

            <li>
              Build and install manually: 

              <table bgcolor="#f0f0f0" cellpadding="10">
                <tr>
                  <td>
<pre>
- Edit src/Configuration:
  &lt;&lt; AddModule    modules/xxxx/mod_foo.o
  &gt;&gt; SharedModule modules/xxxx/mod_foo.so
$ make
$ cp src/xxxx/mod_foo.so /path/to/install/libexec
- Edit /path/to/install/etc/httpd.conf
  &gt;&gt; LoadModule foo_module /path/to/install/libexec/mod_foo.so
</pre>
                  </td>
                </tr>
              </table>
            </li>
          </ul>
        </li>

        <li>
          Build and install a <em>third-party</em> Apache module,
          say <code>mod_foo.c</code>, into its own DSO
          <code>mod_foo.so</code> 

          <ul>
            <li>
              Build and install via <code>configure</code>
              (preferred): 

              <table bgcolor="#f0f0f0" cellpadding="10">
                <tr>
                  <td>
<pre>
$ ./configure --add-module=/path/to/3rdparty/mod_foo.c 
        --enable-shared=foo
$ make install
</pre>
                  </td>
                </tr>
              </table>
            </li>

            <li>
              Build and install manually: 

              <table bgcolor="#f0f0f0" cellpadding="10">
                <tr>
                  <td>
<pre>
$ cp /path/to/3rdparty/mod_foo.c /path/to/apache-1.3/src/modules/extra/
- Edit src/Configuration:
  &gt;&gt; SharedModule modules/extra/mod_foo.so
$ make
$ cp src/xxxx/mod_foo.so /path/to/install/libexec
- Edit /path/to/install/etc/httpd.conf
  &gt;&gt; LoadModule foo_module /path/to/install/libexec/mod_foo.so
</pre>
                  </td>
                </tr>
              </table>
            </li>
          </ul>
        </li>

        <li>
          Build and install a <em>third-party</em> Apache module,
          say <code>mod_foo.c</code>, into its own DSO
          <code>mod_foo.so</code> <em>outside of</em> the Apache
          source tree: 

          <ul>
            <li>
              Build and install via <a
              href="programs/apxs.html">apxs</a>: 

              <table bgcolor="#f0f0f0" cellpadding="10">
                <tr>
                  <td>
<pre>
$ cd /path/to/3rdparty
$ apxs -c mod_foo.c
$ apxs -i -a -n foo mod_foo.so
</pre>
                  </td>
                </tr>
              </table>
            </li>
          </ul>
        </li>
      </ol>

      <h3>Advantages &amp; Disadvantages</h3>

      <p>The above DSO based features of Apache 1.3 have the
      following advantages:</p>

      <ul>
        <li>The server package is more flexible at run-time because
        the actual server process can be assembled at run-time via
        <a
        href="mod/mod_so.html#loadmodule"><code>LoadModule</code></a>
        <code>httpd.conf</code> configuration commands instead of
        <code>Configuration</code> <code>AddModule</code> commands
        at build-time. For instance this way one is able to run
        different server instances (standard &amp; SSL version,
        minimalistic &amp; powered up version [mod_perl, PHP3],
        <em>etc.</em>) with only one Apache installation.</li>

        <li>The server package can be easily extended with
        third-party modules even after installation. This is at
        least a great benefit for vendor package maintainers who
        can create a Apache core package and additional packages
        containing extensions like PHP3, mod_perl, mod_fastcgi,
        <em>etc.</em></li>

        <li>Easier Apache module prototyping because with the
        DSO/<code>apxs</code> pair you can both work outside the
        Apache source tree and only need an <code>apxs -i</code>
        command followed by an <code>apachectl restart</code> to
        bring a new version of your currently developed module into
        the running Apache server.</li>
      </ul>

      <p>DSO has the following disadvantages:</p>

      <ul>
        <li>The DSO mechanism cannot be used on every platform
        because not all operating systems support dynamic loading
        of code into the address space of a program.</li>

        <li>The server is approximately 20% slower at startup time
        because of the symbol resolving overhead the Unix loader
        now has to do.</li>

        <li>The server is approximately 5% slower at execution time
        under some platforms because position independent code
        (PIC) sometimes needs complicated assembler tricks for
        relative addressing which are not necessarily as fast as
        absolute addressing.</li>

        <li>Because DSO modules cannot be linked against other
        DSO-based libraries (<code>ld -lfoo</code>) on all
        platforms (for instance a.out-based platforms usually don't
        provide this functionality while ELF-based platforms do)
        you cannot use the DSO mechanism for all types of modules.
        Or in other words, modules compiled as DSO files are
        restricted to only use symbols from the Apache core, from
        the C library (<code>libc</code>) and all other dynamic or
        static libraries used by the Apache core, or from static
        library archives (<code>libfoo.a</code>) containing
        position independent code. The only chances to use other
        code is to either make sure the Apache core itself already
        contains a reference to it, loading the code yourself via
        <code>dlopen()</code> or enabling the
        <code>SHARED_CHAIN</code> rule while building Apache when
        your platform supports linking DSO files against DSO
        libraries.</li>

        <li>Under some platforms (many SVR4 systems) there is no
        way to force the linker to export all global symbols for
        use in DSO's when linking the Apache httpd executable
        program. But without the visibility of the Apache core
        symbols no standard Apache module could be used as a DSO.
        The only chance here is to use the <code>SHARED_CORE</code>
        feature because this way the global symbols are forced to
        be exported. As a consequence the Apache
        <code>src/Configure</code> script automatically enforces
        <code>SHARED_CORE</code> on these platforms when DSO
        features are used in the <code>Configuration</code> file or
        on the configure command line.</li>
      </ul>
      <!--#include virtual="footer.html" -->
    </blockquote>
  </body>
</html>

